Popularity of cloud deployments has brought forth several challenges. One of the challenges being performance of a software application under different operational workloads. Cloud computing environments, for example Software as a Service (SaaS) may have multiple customers using a common service. Each customer should potentially have unique workload characteristics.
An underlying service may use a set of software programs and/or an architecture for a typical deployment and/or a specific workload. Any variance from the unique workload characteristics shall deteriorate the performance of the software application resulting in service level agreement violations. Often performance enhancements may have been brought by making changes to a source code, an architecture and/or a hardware capacity of the system.
There may have several performance evaluation techniques based on a software development lifecycle. The performance evaluation techniques may include code profiling, stress tests and/or capacity management.
Mutation testing has been used extensively to verify correctness of a software functionality and/or a test suite to be used to test the software application. The source code and/or an assembly code may be changed to mutate the source code and/or the assembly code. The mutated code may semantically be different from the original source code. The mutated code when run may expect to throw an exception, and/or show incorrect results at a point where the mutated code may have been introduced.
Absence of the exception and/or the incorrect results may prove that a part of the source code of the software application may be redundant and/or the test suite may be inadequate. Thus, may not cover the mutated part of the source code. In other words, the mutation testing may be considered as a means to verify completeness and/or correctness of the test suite.
Software Implemented Fault-injections (SWIFI) may relate to injecting faults in the source code and/or into a memory address of the software application just before execution. The SWIFI techniques may be used to test either fault tolerance of the system, and/or study failure modes. The SWIFI may operate with the assembly code, where at run-time various instructions and/or data may be manipulated to change randomly and/or in a pre-defined location. Such a manipulation may be a single bit and/or a multi bit flip.
The source code of a software application written in high level languages such as COBOL, C, C++, JAVA may have become prevalent. There may have been various consequences of providing the source code for testing the software application. Intellectual property (IP) risk may have been main concern in organizations. There may have the source code changes during the testing of the software application. Bugs may also be added during the testing of the software application.